Comparing a TBL Tagger with an HMM Tagger: Time Efficiency, Accuracy, Unknown Words

In this paper a transformation-based learning tagger is compared with a hidden Markov model tagger. For this comparison the Brill tagger and the TnT tagger are used. The Dutch Spoken Corpus (CGN), tagged with a medium-sized (72) tagstet, is used as training and testing material. The TnT tagger outperforms the Brill tagger on larger tagsets and when relatively small training-sets (around 10.000 sentences) are used. The results support the observation by Schneider and Volk that the unknownword handling of the Brill tagger is still unsatisfactory. While both taggers achieve around the same tagging speed the Brill-tagger takes a long time to train. The Brill tagger takes around 18 hours to train for a training-set of 100.000 sentences tagged with the 72-tags tagset; the TnT tagger takes slightly less than nine seconds for that and slightly less then a minute and a half to train on a set of 900.000 sentences. The best result for the Brill-tagger on the medium-sized tagset was an error-rate of 3.9% with an training-set of 100.000 sentences. The best result for the TnT-tagger on the medium-sized tagset was an error-rate of 2.7% with an training-set of 900.000 sentences.